Skip to content

Message Attestation prototype implementation#1

Draft
liustanley wants to merge 23 commits into
mainfrom
jaime/opamp-message-attestation
Draft

Message Attestation prototype implementation#1
liustanley wants to merge 23 commits into
mainfrom
jaime/opamp-message-attestation

Conversation

@liustanley

@liustanley liustanley commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Message Attestation prototype — Client + Server

Spec PR: open-telemetry/opamp-spec#333
Status: Draft — awaiting maintainer review of the signing/ package before merging


Summary

This PR implements the Message Attestation feature from the OpAMP specification. Every ServerToAgent message is wrapped in a SignedServerToAgent envelope carrying a detached X.509 signature. Agents that opt in can verify that responses originate from an authorised distribution server and have not been tampered with in transit — even when TLS is terminated at a load balancer.

The feature is fully opt-in and backward-compatible: agents and servers that do not set the new capability bits continue to use the existing wire format unchanged.


What changed

New signing/ package

The core cryptographic layer. All types are defined as interfaces so operators can swap in HSMs, remote signing services, or custom trust stores without changing the OpAMP client/server code.

File Purpose
types.go Signer, Verifier, TrustAnchorProvider, TOFUStore interfaces; Algorithm enum
algorithm.go Algorithm detection from leaf's subjectPublicKeyInfo; sign/verify dispatch for all four baseline algorithms
chain.go RFC 5280 §6 path validation; enforces ExtKeyUsageCodeSigning to prevent TLS certs from being repurposed as signing certs
local_signer.go In-process signer backed by a crypto.Signer; optional WithRootCA for TOFU enrollment
local_verifier.go In-process verifier backed by an *x509.CertPool
loader.go VerifierFromFile, VerifierFromPEM, LocalSignerFromFiles for config-file wiring
tofu.go FileTOFUStore — persists the enrolled root CA with O_EXCL for idempotent writes
remote_signer.go RemoteSigner — delegates Sign/ChainDER/TrustAnchorPEM to an HTTP policy server
certs.go GenerateCA/GenerateLeaf test helpers with CertOptions (SANs, validity windows)

Server-side wiring (server/)

  • attestation.goconnectionSigningState snapshots the cert chain at connection open (freezes operator-side rotation for the connection lifetime), signs every outbound ServerToAgent, and delivers the chain exactly once per WS connection via atomic.CompareAndSwap. TOFU failures are delivered as TrustChainResponse.error_message rather than silently dropping the connection.
  • serverimpl.goSettings.PayloadSigner is the attestation entry point. When set, the server auto-advertises OffersPayloadTrustVerification in every response and wraps outbound messages after capability negotiation. The HTTP path re-delivers the chain on every request (stateless transport).
  • wsconnection.goErrSendBeforeNegotiated guard prevents Send being called before the agent's capabilities are known. requiresNegotiation/signing state is held per-connection.

Client-side wiring (client/)

  • client/internal/attestation.goattestationState per-connection: validates the chain on the first message (including SAN check via leaf.VerifyHostname), verifies the signature on every message. TOFU enrollment bootstraps the verifier from TrustChainResponse.tofu_trust_anchor and persists it via TOFUStore.Save. Reset() supports the HTTP polling transport.
  • wsclient.gorunUntilStopped applies a separate backoff.NewExponentialBackOff() for attestation failures so the client does not spin-reconnect against a server it just rejected.
  • client/internal/httpsender.go — same exponential backoff pattern for the HTTP polling transport; attestationState.Reset() is called on failure to allow recovery on the next poll.
  • client/types/startsettings.goPayloadVerifier signing.Verifier and PayloadTOFUStore signing.TOFUStore fields (mutually exclusive).

Out-of-process policy server example (internal/examples/policysrv/)

Demonstrates the recommended production deployment pattern: the OpAMP distribution server holds no private key material. Signing is delegated to a separate policy server that can decode the payload, enforce organisational policies (team ownership, allowed message types), and delegate to an HSM before returning a signature.

  • policysrv/main.go — HTTP server with /v1/sign, /v1/chain, /v1/ca endpoints
  • signing/remote_signer.go — client stub implementing Signer + TrustAnchorProvider
  • Agent binary gains --attestation-ca and --attestation-tofu-store flags (mutually exclusive)

Protobuf regeneration

protobufs/opamp.pb.go regenerated from the updated spec proto:

  • SignedServerToAgent message (payload + signature + trust_chain_response)
  • TrustChainResponse message (certificate_chain + error_message + tofu_trust_anchor)
  • AgentCapabilities_RequiresPayloadTrustVerification
  • AgentCapabilities_AcceptsPayloadTrustAnchorTOFU
  • ServerCapabilities_OffersPayloadTrustVerification

Security properties

Property How it's enforced
Server authenticity X.509 chain validated against operator-provisioned trust anchor; ExtKeyUsageCodeSigning prevents TLS cert reuse
Message integrity Detached signature over marshalled wire bytes verified on every message
Server binding leaf.VerifyHostname(serverName) after chain validation
TOFU immutability FileTOFUStore.Save uses O_CREATE|O_EXCL; enrolled anchor cannot be updated in-band
Downgrade protection Empty inner ServerToAgent (field-1 wire-type collision) is rejected with ErrEmptyInnerServerToAgent
Compromised server isolation Attestation failure terminates the WS connection immediately; HTTP path resets state and applies exponential backoff

Revocation checking (CRL/OCSP) is not implemented in this prototype. Operators should use short-lived leaf certificates in the interim.


Known limitations / follow-up work

  • CRL/OCSP revocation checking in signing/chain.go
  • FileTOFUStore has no cross-process file locking (single-agent-per-host is the expected deployment)
  • Collector opampextension and Supervisor wiring (separate PRs, pending Evan Bradley confirmation)

Testing

  • signing/: unit tests for all four algorithms, adversarial inputs, chain validation edge cases, TOFU store idempotency, benchmarks (BenchmarkSign/BenchmarkVerify)
  • server/: unit tests for connectionSigningState first-send semantics, concurrent send, capability bit helpers, TOFU error path; WS end-to-end tests for the wrap/no-wrap negotiation matrix
  • client/internal/: unit tests for attestationState (first message, subsequent messages, TOFU enrollment, SAN mismatch, downgrade attack, error propagation)
  • internal/integrationtest/: full end-to-end tests over a real WS connection covering all four algorithms, mid-stream server key rotation recovery, TOFU enrollment

truthbk and others added 23 commits May 19, 2026 17:33
Re-point internal/opamp-spec submodule at truthbk/opamp-spec branch
jaime/x509-spec-full-protocol (commit 65b74b8) and regenerate
protobufs/. This pulls in the wire-level additions for OpAMP Message
Attestation:

* ServerToAgent.trust_chain_response = 12 (TrustChainResponse)
* ServerToAgent.signature = 13 (bytes)
* AgentCapabilities_RequiresPayloadTrustVerification = 0x00010000
* ServerCapabilities_OffersPayloadTrustVerification = 0x00000080
* New TrustChainResponse message (with nested Certificate)

Generated locally with protoc 34.1 + protoc-gen-go (native arm64) to
work around a SIGSEGV in the otel/build-protobuf Docker image under
amd64 emulation on Apple Silicon. Output is wire-compatible with the
older generator; the diff is mostly newer protoc-gen-go's more
compact table-driven reflection style.

No source-level call-site changes required: existing accessor methods
and enum constants are preserved.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bump internal/opamp-spec submodule to commit 1f75ca1 (the Phase A spec
pivot to detached signing via SignedServerToAgent envelope) and
regenerate protobufs/opamp.pb.go.

Wire-level changes vs the previous regen (e21eeda):

* ServerToAgent.TrustChainResponse (field 12) removed
* ServerToAgent.Signature (field 13) removed
* Both field numbers reserved on ServerToAgent (cannot be reused)
* New SignedServerToAgent message added:
    Payload            []byte (field 1)
    Signature          []byte (field 2)
    TrustChainResponse *TrustChainResponse (field 3)

ServerToAgent is now byte-identical to upstream OpAMP. The
SignedServerToAgent envelope appears on the wire only when both
peers have negotiated payload trust verification.

Generated locally with protoc 34.1 + protoc-gen-go (native arm64) to
work around a SIGSEGV in the otel/build-protobuf Docker image under
amd64 emulation on Apple Silicon.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce the opamp-go signing/ package, the foundation for Message
Attestation in the client and server wire layers (Phases 3-4).

Public API:

* Signer interface  — Sign(ctx, payload []byte) → ([]byte, error)
                      ChainDER(ctx) → ([][]byte, error)
* Verifier interface — ValidateChain(ctx, chainDER, now) → (*x509.Certificate, error)
                      Verify(ctx, payload, signature, leaf) → error
* LocalSigner       — in-process reference Signer, holds a crypto.Signer
                      private key and a DER cert chain.
* LocalVerifier     — in-process reference Verifier, wraps an
                      x509.CertPool trust anchor pool.
* VerifierFromFile  — convenience constructor reading a PEM CA bundle.
* LocalSignerFromFiles — constructor reading PEM-encoded leaf key + chain.
* GenerateCA / GenerateLeaf — multi-algorithm cert generation for tests
                              and example servers (CertOptions for clock
                              skew + custom validity windows).

Algorithm dispatch covers the spec baseline:
* ECDSA P-256 + SHA-256 (DER-encoded (r,s))
* ECDSA P-384 + SHA-384
* RSA-2048+ with PKCS#1 v1.5 + SHA-256
* Ed25519

Path validation uses x509.Certificate.Verify with the
ExtKeyUsageCodeSigning EKU enforced. crypto/x509 handles signature,
validity window, basicConstraints, pathLenConstraint, and critical
extension checks.

The Signer interface is deliberately minimal so RPC-backed signers
(e.g. Datadog rc-x509-api) can be plugged in without touching the
wire-level opamp-go code. The signer takes raw payload bytes; the
opamp-go server marshals the inner ServerToAgent and hands those bytes
to Sign. No proto knowledge in this package.

Tests cover:
* Per-algorithm sign/verify round-trip (4 algorithms).
* Tampered signature → ErrSignatureMismatch.
* Tampered payload   → ErrSignatureMismatch.
* Expired leaf, not-yet-valid leaf, unknown trust anchor, garbage cert
  bytes, leaf with wrong EKU (TLS server auth instead of code signing).
* PEM file round-trips for both the verifier and the signer, in all
  four algorithms.
* Context cancellation propagation through every method.
* LocalSigner.ChainDER defensive-copy semantics.

go test ./signing/... -race -count=1 passes; go vet clean; gofumpt
clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four issues surfaced by review of the signing/ package:

1. algorithmFromCert dispatched from cert.SignatureAlgorithm (the
   *issuer's* signing algorithm) rather than from the leaf's actual
   public key type and curve. Within-family mismatches (e.g. a P-384
   CA issuing a P-256 leaf with SignatureAlgorithm=ECDSAWithSHA384)
   could slip past type assertions because both keys are
   *ecdsa.PrivateKey/PublicKey. Fix: dispatch on leaf.PublicKey type
   and ECDSA curve; cross-check that cert.SignatureAlgorithm is
   consistent with the pubkey.

2. No minimum RSA modulus enforced. The docstring claimed "2048-bit
   minimum recommended" but nothing checked it. Add a constant
   rsaMinModulusBits = 2048 and reject smaller keys in
   algorithmFromCert.

3. parsePrivateKeyPEM had a dead default branch: if PKCS#8 parsed but
   produced a non-Signer type, it returned an error instead of
   falling through to PKCS#1/EC. Simplified to a crypto.Signer
   interface assertion (all stdlib private-key types satisfy it).
   Drop the now-unused crypto/ecdsa, crypto/ed25519, crypto/rsa
   imports.

4. New adversarial test file (signing/adversarial_test.go) covers
   the high-risk paths the review flagged:
   - TestVerify_LeafPublicKeyMismatch: sig from key A vs leaf with
     pubkey B → ErrSignatureMismatch
   - TestVerify_AlgorithmFamilyMismatch: ECDSA sig vs RSA leaf
   - TestWrongChainOrder_DetectedAtSignatureVerify: documents that
     X.509 path validation accepts a chain with [leaf, intermediate]
     ordering (Go's x509 is permissive about EKU absence on the
     intermediate), but the per-message signature step catches it.
     This is the load-bearing protection.
   - TestValidateChain_IntermediateNotSignedByRoot: attacker-rooted
     chain rejected
   - TestNewLocalSigner_LeafPublicKeyMismatchDetected: documents the
     deliberate non-validation at construction time, asserts the
     observable symptom (Sign + Verify round-trip fails) and provides
     a regression hook if we choose to validate at construction later
   - TestParseCertChainPEM_IgnoresNonCertificateBlocks: covers the
     "skip non-CERTIFICATE block" branch in parseCertChainPEM
   - TestAlgorithmFromCert_RSAKeyTooSmall: 1024-bit RSA rejected
   - TestAlgorithmFromCert_AlgorithmDeclarationMismatch: P-256 key
     with declared SHA-384 signature algorithm rejected

go test ./signing/... -race -count=1 passes. go vet + gofumpt clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three small follow-ups from the second code-review pass:

1. Stale docstrings in local_signer.go, local_verifier.go, and types.go
   still described the algorithm as "derived from
   leaf.SignatureAlgorithm". The 94c621e refactor changed the
   authority to leaf.PublicKey with SignatureAlgorithm cross-checked;
   update the docs so future readers (and any RPC-backed Signer
   implementer) don't re-introduce the original bug.

2. TestVerify_AlgorithmFamilyMismatch used a permissive
   `errors.Is(ErrSignatureMismatch) || errors.Is(ErrUnsupportedAlgorithm)`
   assertion. The outcome is actually deterministic: the verifier
   reaches verifyWithPub's RSA branch with ECDSA-DER bytes, which
   always returns ErrSignatureMismatch. Tighten to a single ErrorIs.

3. TestParseCertChainPEM_IgnoresNonCertificateBlocks had dead setup
   (writing a chain file to disk that was never read). Drop the file
   I/O. Also strengthen the test by using a well-formed PRIVATE KEY
   PEM block as the "junk" entry (the realistic mis-bundling case)
   instead of a PUBLIC KEY block with 3 garbage bytes.

Drop unused imports (errors, os, path/filepath) from adversarial_test.go.

go test ./signing/... -race -count=1 passes; go vet clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3 of the OpAMP Message Attestation implementation: client-side
wiring for the negotiated payload-trust verification path.

When the Agent opts in by configuring StartSettings.PayloadVerifier
and setting AgentCapabilities_RequiresPayloadTrustVerification, every
inbound server-to-agent message is treated as a SignedServerToAgent
envelope:

- First message carries trust_chain_response. The agent validates the
  delivered chain against its pre-configured trust anchor pool and
  stores the resulting leaf for the connection's lifetime. The
  signature MAY be empty on this first message; if present, it's
  verified as defence-in-depth.
- Subsequent messages MUST carry a detached signature over the
  payload bytes. The agent verifies against the stored leaf.
- Any failure (missing chain, bad chain, missing/invalid signature)
  returns an error from the receive path, terminating the WebSocket
  receive loop (and therefore the connection). HTTP polling logs and
  skips the response; the next poll re-handshakes.

When PayloadVerifier is nil (the default), the wire format is
byte-identical to upstream OpAMP — strict opt-in at the wire level.

Architecture:

- client/internal/attestation.go (new): per-connection
  attestationState handles the handshake + per-message verification.
  unwrapServerToAgent acts as either a pass-through (nil state) or an
  envelope unwrap.
- client/types/startsettings.go: new PayloadVerifier signing.Verifier
  field.
- client/internal/clientcommon.go: stores verifier on ClientCommon
  before validateCapabilities runs; new consistency check ensures the
  Requires bit ↔ verifier presence.
- client/internal/wsreceiver.go: NewWSReceiver gains a
  payloadVerifier signing.Verifier parameter; receiveMessage now
  threads context and strips the WS framing first, then dispatches to
  unwrapServerToAgent.
- client/internal/httpsender.go: HTTPSender.Run gains a
  payloadVerifier parameter; receiveResponse uses unwrapServerToAgent.
- internal/wsmessage.go: factor StripWSMessageHeader out of
  DecodeWSMessage so callers can strip the framing without committing
  to a proto message type at that moment.

Tests (17 new tests, all passing under -race):

- attestation_test.go: ProcessEnvelope happy path + 8 reject paths
  (missing chain, error_message reported, unknown CA, missing
  signature on subsequent, tampered signature, empty payload, tampered
  first signature). unwrapServerToAgent pass-through + envelope.
- clientcommon_attestation_test.go: capability ↔ verifier consistency
  (four quadrants).

Pre-existing TestRedirectHTTP failures and three pre-existing vet
warnings on the client/internal/packagessyncer.go path remain
unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups from the Phase 3 code review:

1. WebSocket: eagerly close the connection on attestation failure
   (wsreceiver.go). The receive loop already returned on
   attestation-related errors but never invoked r.conn.Close(),
   leaving a small window where the sender goroutine could keep
   writing AgentToServer messages to an untrusted server before the
   wsclient owner observed the stopped signal and tore the connection
   down. Per the Message Attestation spec the Agent MUST terminate
   the connection on any payload-trust verification failure; this
   commit closes the conn eagerly when isAttestationFailure(err)
   classifies the error as attestation-related.

   New helper isAttestationFailure() in attestation.go centralises
   the errors.Is checks against the local sentinels (ErrMissingTrust
   Chain, ErrTrustChainErrorReported, ErrMissingSignature, ErrMissing
   Payload) plus the signing-package sentinels propagated up (ErrChain
   Validation, ErrSignatureMismatch, ErrEmptyChain, ErrParseCertificate,
   ErrUnsupportedAlgorithm).

2. HTTP: reset per-connection attestation state on verification
   failure (httpsender.go::receiveResponse). The previous behaviour
   created the attestationState once in Run() and never reset it; an
   attestation failure after the initial successful handshake (e.g.
   following server-side key rotation) would keep firstSeen=true and
   the cached leaf, causing every subsequent poll to fail because the
   server's new chain handshake would be treated as a normal signed
   message and rejected for missing signature.

   New attestationState.Reset() method clears firstSeen + leaf under
   the mutex. HTTP receiveResponse now calls it on any unwrap failure
   so the next poll can re-attempt the handshake. WebSocket callers
   do not need Reset — failure terminates the connection and the
   next reconnect attempt constructs a fresh attestationState.

Tests:
* TestAttestationState_Reset: drives a successful handshake, calls
  Reset, asserts firstSeen/leaf cleared and that the next envelope
  without trust_chain_response now produces ErrMissingTrustChain
  rather than ErrMissingSignature, then verifies a fresh first
  envelope succeeds.
* TestIsAttestationFailure_ClassifiesSentinels: covers the
  classification helper across all sentinels (local + signing
  package), confirms wrapped errors still classify via errors.Is,
  and confirms generic transport errors do NOT classify.

go test ./client/internal/... -race -count=1 passes; pre-existing
TestRedirectHTTP failure on upstream/main and pre-existing vet
warnings on the client/internal/packagessyncer.go family unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 of the OpAMP Message Attestation implementation: server-side
wiring for the payload-trust verification path.

When the operator configures Settings.PayloadSigner and a connecting
Agent declares AgentCapabilities_RequiresPayloadTrustVerification on
its first AgentToServer, the Server wraps every outbound
ServerToAgent in a SignedServerToAgent envelope per the spec:

- The first envelope on a WebSocket connection carries
  trust_chain_response (the cert chain delivered by Signer.ChainDER
  at accept time) and a detached signature over the payload bytes.
- Subsequent envelopes on the same connection carry the signature
  only. The chain is snapshotted at connection accept so mid-stream
  signer-side rotation does not affect a live connection (the agent
  re-handshakes on reconnect).
- For HTTP polling (no persistent connection), every response carries
  both the chain and a signature. The client's per-HTTPSender
  attestation state ignores the chain after the first.

ServerCapabilities_OffersPayloadTrustVerification is auto-set on
outgoing capabilities when PayloadSigner is configured for the
connection. Operators don't need to remember to OR it in.

When PayloadSigner is nil (the default) or the Agent did not declare
Requires, the wire format is byte-identical to upstream OpAMP.

Architecture:

- server/server.go: Settings.PayloadSigner signing.Signer field.
- server/attestation.go (new): connectionSigningState with
  signOutgoing(ctx, *ServerToAgent) → *SignedServerToAgent; first
  call includes trust_chain_response, subsequent calls don't.
  Helpers agentRequiresAttestation() and addOffersAttestationBit().
- server/wsconnection.go: wsConnection gains a *connectionSigningState
  field + enableSigning() method. Send() wraps if signing is enabled.
- server/serverimpl.go::handleWSConnection: after reading the first
  AgentToServer, if the Agent declares Requires AND PayloadSigner is
  configured, snapshot the chain and enable signing on the
  wsConnection. Auto-OR the Offers bit on every outbound response.
- server/serverimpl.go::handlePlainHTTPRequest: per-response chain
  snapshot + sign + wrap when the Agent declares Requires.

Tests (8 new, all passing under -race):

- connectionSigningState_FirstAndSubsequent: first envelope carries
  chain + signature, subsequent only signature; both verify against
  paired LocalVerifier.
- connectionSigningState_ConcurrentFirstSend: exactly one of two
  concurrent signOutgoing calls carries the chain (firstSent guarded).
- connectionSigningState_NilSigner / _SignerError: error paths from
  newConnectionSigningState.
- agentRequiresAttestation / addOffersAttestationBit: helpers.
- ServerWraps_WhenAgentRequires_WS: WS integration — agent sets
  Requires on first AgentToServer; server response decodes as
  SignedServerToAgent; signature verifies; OffersPayloadTrustVerification
  is auto-set on the inner ServerToAgent's capabilities.
- ServerDoesNotWrap_WhenAgentDoesNotRequire_WS: agent omits Requires;
  server's response carries no signature, no chain. Strict opt-in
  preserved.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Covers the full client+server signing round trip across all four
supported algorithms (ECDSA P-256/P-384, RSA-PKCS#1v1.5 SHA-256,
Ed25519) on the WebSocket transport, plus an HTTP transport sanity
check. Reject scenarios exercised end-to-end: server has no signer,
expired leaf, unknown CA, tampered subsequent signature, HTTP under
unknown CA.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- OffersPayloadTrustVerification is now advertised whenever the server
  has a PayloadSigner configured, independent of any individual agent's
  Requires bit. This matches the spec's negotiation matrix (No/Yes
  quadrant: server is capable, agent didn't opt in) and the
  Settings.PayloadSigner docstring.
- wsConnection.signing is now an atomic.Pointer so Send is safe to call
  from a goroutine other than the receive loop (matches the public
  Connection callback contract).
- Send before the first AgentToServer is rejected with
  ErrSendBeforeNegotiated when PayloadSigner is configured, closing the
  silent-bypass window where OnConnected callbacks could emit unsigned
  bytes before the agent's capability bits were seen.
- connectionSigningState.firstSent is now an atomic.Bool +
  CompareAndSwap; drops the per-state mutex.
- handleWSConnection's signingNegotiated local is replaced by the
  connection's negotiated atomic.Bool, accessed via isNegotiated /
  markNegotiated accessors.
- HTTP path notes a TODO for v2 ChainDER caching (deferred — affects
  RPC-backed signers only).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 5 e2e suite:
- Raise e2eDeadline (5s → 15s) and e2eNonOccurrenceDeadline (750ms →
  1.5s) so tests don't flake on -race-loaded CI.
- Replace assert.Eventually with require.Eventually on positive
  awaits that gate subsequent dereferences (eliminates the
  nil-pointer SIGSEGV the agent flagged in the tampered-signature
  test on timeout).
- Extract assertRejected helper that all four reject scenarios now
  share — drops ~100 lines of duplication.
- Rename tamperingSigner → controlledSigner and add failFromCall mode
  so it can drive both tampered-signature and mid-stream Sign failure
  tests. Tampering now copies the slice before mutating (no inner
  signer coupling).
- Drop the brittle HTTP-specific "cannot unmarshal response" log
  assertion in favour of the canonical "Payload trust verification
  failed" sentinel — httpsender.go now emits that phrase whenever
  isAttestationFailure(err) is true, matching the WS receive path.

New e2e coverage:
- TestE2E_ConcurrentConnections_MixedSigningState — two agents on one
  server, one with Requires + verifier, one without; confirms
  per-connection signing state is isolated.
- TestE2E_Reject_MidStreamSignFailure_WS — exercises the
  controlledSigner.failFromCall path end-to-end.
- TestE2E_SendBeforeNegotiation_Errors_WS — confirms the new
  ErrSendBeforeNegotiated guard from wsConnection.Send when
  PayloadSigner is configured and OnConnected pushes prematurely.
- TestE2E_SendBeforeNegotiation_NoSigner_AllowsSend_WS — control case
  confirming the guard is scoped to attestation-enabled servers.

Phase 4 unit tests:
- TestServerDoesNotWrap_WhenAgentDoesNotRequire_WS now asserts the
  Offers bit IS advertised in the response, matching the new spec
  semantics (server is capable; agent declined).
- TestSignOutgoing_MidStreamSignFailure_PropagatesError wires up the
  previously-dead failingSigner.signErr field.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TestRedirectHTTP/{simple_redirect,check_redirect} have been failing on
upstream master since open-telemetry#269 introduced them — this is a pre-existing bug
unrelated to Message Attestation work; we are fixing it here so the
client/ suite goes green.

Two bugs combined to silently break the test:

1) `redirectServer` (client/wsclient_test.go) accepted a status int
   argument but hardcoded http.StatusSeeOther (303), so the test
   intent of "redirect with 302" actually emitted 303 on the wire.

2) Both 302 and 303 cause net/http's client to convert POST → GET when
   following the redirect (RFC 7231 §6.4.4), stripping Content-Type
   along the way. The OpAMP mock server dispatches on
   Content-Type: application/x-protobuf and falls through to a
   WebSocket upgrade for anything else — the upgrader writes 400 on a
   GET without WS headers, which the HTTP client treats as a
   non-retryable error and silently drops without firing
   OnConnect/OnConnectFailed. The test's eventually(...) wait then
   times out.

Fix:
- Honour the status arg in redirectServer (one-line cleanup).
- Switch TestRedirectHTTP to http.StatusTemporaryRedirect (307), which
  preserves both method and body across the redirect, so the
  redirected request stays a POST with the protobuf payload and the
  mock server's plain-HTTP path handles it normally.
- Set req.GetBody in HTTPSender.prepareRequest so net/http can replay
  the request body on method-preserving redirects (307/308) without
  the "http: can't replay request body" error. This is also a small
  real fix for OpAMP HTTP clients pointing at servers behind
  redirecting load balancers.

TestRedirectWS continues to use 302 — WebSocket uses GET, so the
POST→GET conversion is a no-op and the redirect chain works either way.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
signing/doc.go (new)
- Moved the 23-line package-comment block out of types.go into its
  own doc.go file, matching the Go convention for package
  documentation. Added a paragraph noting that GenerateCA /
  GenerateLeaf are exported test helpers (kept in this package
  rather than a sibling signingtest subpackage because in-package
  signing tests need them without creating an import cycle).
- signing/types.go now opens directly with `package signing`.

Defense-in-depth on the client receive path:
- client/internal/attestation.go's unwrapServerToAgent now rejects
  inner ServerToAgent payloads that decode to all-default values
  (proto.Equal to the zero ServerToAgent). New sentinel
  ErrEmptyInnerServerToAgent + entry in isAttestationFailure.

  Rationale: proto3 field-1 wire-type collision means a malicious
  server that downgrades by sending a plain ServerToAgent (no
  envelope) has its InstanceUid bytes misinterpreted as
  SignedServerToAgent.payload. The inner decode of those random
  UUID bytes either errors (proto3 tag validation) or produces a
  default-valued ServerToAgent. ProcessEnvelope's chain/signature
  checks already terminate the connection on the first message of
  this attack — this check is belt-and-suspenders that pins the
  contract: every legitimate ServerToAgent the agent processes has
  at least one non-default field (typically InstanceUid, auto-filled
  by serverimpl.go).

Verified clean under -race × 2 across signing/, client/internal/,
internal/integrationtest/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Gives operators a number when they ask "what's the per-message overhead
of OpAMP Message Attestation?" — and gives us a regression signal
against the algorithm dispatch code.

Measurements on Apple M1 Max (10 iterations each):

  BenchmarkSign/ECDSA-P256-SHA256       25 µs / 6.1 KB / 59 allocs
  BenchmarkSign/ECDSA-P384-SHA384      174 µs / 6.2 KB / 61 allocs
  BenchmarkSign/RSA-PKCS1v15-SHA256   1264 µs / 512 B  /  2 allocs
  BenchmarkSign/Ed25519                 31 µs /  88 B  /  1 alloc

  BenchmarkVerify/ECDSA-P256-SHA256     87 µs /  576 B / 10 allocs
  BenchmarkVerify/ECDSA-P384-SHA384    606 µs /  808 B / 17 allocs
  BenchmarkVerify/RSA-PKCS1v15-SHA256   30 µs / 1376 B /  9 allocs
  BenchmarkVerify/Ed25519               45 µs /    0 B /  0 allocs

(Asymmetric Sign/Verify costs are expected — RSA verify is cheap but
sign is expensive; ECDSA is the opposite. Ed25519 is fast and
allocation-free on both sides, making it the right default for both
small and large fleets.)

Run with:  go test -bench=. -benchmem ./signing/

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pins a contract that was documented only in code comments before:
rotating the server's underlying signing key mid-stream MUST NOT
compromise the agent's currently-live connection — signatures
produced by the rotated key fail verification against the
snapshotted leaf the agent locked in on handshake, the agent
terminates the connection, and on the next reconnect it re-fetches
the new chain.

Two locations in opamp-go already promise this behaviour in
comments:
- server/attestation.go's connectionSigningState: "the certificate
  chain is snapshotted at construction time so that operator-side
  cert rotation does not affect a live connection"
- client/internal/httpsender.go's Reset(): "recover from mid-stream
  faults such as server-side key rotation"

Neither claim had a test until now.

The new TestE2E_MidStreamKeyRotation_DoesNotAffectLiveConnection
uses a rotatableSigner (atomic.Pointer to *LocalSigner) so the test
can swap the underlying signer at any point without re-running the
server's chain-snapshot path. After the swap, signatures use the
new key but the agent's cached leaf is still the old one — verifying
fails and the connection terminates.

Verified clean under -race × 3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…essage signature

Switch TrustChainResponse.certificate_chain from a repeated DER-encoded
Certificate message to a single PEM blob, matching the format now
specified in the spec branch (truthbk:jaime/x509-spec-full-protocol).
The proto is regenerated from the updated spec proto file; the nested
TrustChainResponse_Certificate type is removed.

On the server side, the snapshotted DER chain is now encoded to a
concatenated PEM blob before being placed in TrustChainResponse.
On the client side, the PEM blob is decoded back to individual DER
slices via a new parsePEMChain helper before being passed to
Verifier.ValidateChain.

Make the signature field mandatory on every SignedServerToAgent,
including the first message. The spec removed the "MAY leave signature
empty on the first envelope" exception; ProcessEnvelope now enforces
signature presence and validity unconditionally after chain validation,
regardless of whether it is the first or a subsequent message.

Tests are updated throughout: PEM encoding replaces DER construction in
test fixtures, all happy-path calls to buildFirstEnvelope pass
signFirst=true, and a new TestAttestationState_MissingSignatureOnFirst
test covers the first-message rejection path.
Introduces a self-contained three-component demo of the attestation
architecture described in supplementary-guidelines.md.

policysrv (new): a minimal out-of-process policy/signing server that
generates an ephemeral ECDSA P-256 CA and leaf on startup, writes the CA
to /tmp/opamp-policy-ca.pem for agents to use as their trust anchor, and
exposes POST /v1/sign, GET /v1/chain, and GET /v1/ca. The distribution
server delegates all signing here and never holds the private key.

signing.RemoteSigner (new): implements signing.Signer by posting payloads
to /v1/sign and fetching the certificate chain from /v1/chain, keeping the
key isolation boundary at the HTTP layer.

server example: gains --policy-server to construct a RemoteSigner and pass
it as PayloadSigner into server.Settings, enabling signed ServerToAgent
delivery without the distribution server touching key material.

agent example: gains --attestation-ca (env: AGENT_ATTESTATION_CA) to load
a trust anchor via signing.VerifierFromFile, store it with
WithPayloadVerifier, advertise RequiresPayloadTrustVerification, and
forward PayloadVerifier into types.StartSettings so every inbound message
is verified by the client layer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants